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ABSTRACT 



The production and sale of instructional materials 
are now big business in this country,. Hence it is desirable, and 
probably necessary, to establish standards for such materials. 
Research in the area of curriculum materials is basically virgin 
territory. The evaluation of different types of curriculum materials 
will require the application of different types of criteria. Some 
suggestions regarding the distinctions between types of materials are 
made. With respect to curriculum materials which aim at the 
attainment of specific objectives, the educational significance of 
content, flexibility, durability, incidence of use by both teachers 
and students, ease of use, and educational effect are proposed as 
possible assessment criteria. (DG) 
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Although curriculum materials in a general sense have been used by 
teachers since the time teaching began, the Jiarge-scaie production and sale of 
curriculum materials is a recent ; development, a child of the curriculum de** 
veiopment movement. That movement, which started in 1952 with Max Beberman's 

work in mathematics at the University of Illinois now has a history spanning el- 

♦ 

most two decades. During that period we have seen curriculum development 
projects emerge in almost all fields of study including the arts. With the 
support of die National Science Foundation and the U, S. Office of Education 
projects in science and mathematics have received well over 100 million dollars 
In support over the past ten years, A significant portion of that amount as well 
as portions of the support provided for projects la the social studies and the arts 
are being spent on the development of material especially designed to accompany 
the new curricula. 

If one thinks back to the more typical modes of curriculum making two 
patterns emerge as stark contrasts to the large-scale project approach that has 
characterized the past educational decade. One pattern Is that of providing 
teachers with textbooks and leaving it to diem to beg, borrow, steal or construct 

♦Presented at the American Educational Research Conference 
Minneapolis, Minnesota, March, 1970. 
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the resources necessary for working with children in a particular unit of study. 

The of the elementary school teacher scurrying about to rescue old 

bags, cans, string. Jars, wire and so forth with which to teach a lesson is 

neither false nor completely faded. The teacher, especially at the elementary 
+ 

level, was and is respected to improvise with found objects. 

The second pattern is one of providing a curriculum guide but little 

else. In the first pattern, not even a guide is provided. The teacher receives 
the textbook but develops die educational program on her own. In the second 
pattern, the textbook and the guide arc provided. The materials are not provided 

although they may be suggested. 

Contrast these patterns with the well -designed, smartly -packaged, 
fully-integrated, multi-media kits provided by SRA, Heath and Co., Scott Foresma n , 
rw.,.1 Learning, and so forth, and it t/lll be easy to appreciate the difference In 
between the old and the new approaches to curriculum making. With the 
availability of funds and the assumption that curriculum could be developed best 
not by the teacher but by specialized professional curriculum writers working 
under the guidance of an academic scholar and supported by the services of a 
profit-making corporation has come a host of new curricula and curriculum 

materials* 

Now whenever programs developed in educational contexts remain small 
or are believed Impotent the need for monitoring or for formulating standards by 
which to appraise them is similarly small. We tend not to worry about those things 
that we believe to be feckless. It is understandable therefore that interest has been 



developing to find ways to review and appraise the quality of the curriculum 
materials that are being sold to school districts throughout the country* 1 
When a field or a practice becomes important to the perceived welfare of a 
community, social controls are instituted to be sure that the community’s 
welfare is protected. The desire to establish standards for curriculum materials 
is a reflection of such concern. And too it signals one of the hallmarks of a 
responsible profession -*«a willingness to monitor the quality of die service it 
affords its clients. 

There is of course another good reason to establish criterial for the 
review of curriculum materials. The production and sale of such materials is 
big business. And business in America has not demonstrated marked response , 
bility in serving the public’s interest. There are, of course, some exceptions 
but it would be unwise for educational scholars to rely on the merchant to 
monitor the quality of his own products, especially when die consumers are 

children. Thus we have a situation where for the first time in American educa** 

# * 

tional history the field of education is beginning to think seriously about ways and 
means of developing and applying standards to ensure the quality of the educational 
materials that children in this country’s schools will use. We are apparently 
coming to a point where it will be possible tv review critically the materials that 
are sold to schools in light of their educational value. 

With such a vision of the desirable comes many and severe problems, 
for although die desire is laudable, die means for realizing it are not especially 
clear, Louise Tyler and Frances Kline have taken very important first steps 
but there is much to be done. For example, conceptual clarification regarding 
types of curriculum materials and dicir modes of use have yet to be made. 
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Research on the utility and satisfaction teachers and students have with v istructional 
materials are exceedingly hard to find in the literature. After searching five 
major journals in which such research might be found 1 was able to locate only 
four studies which are, at best, only remotely related to the hinds of questions 
that researchers can ask of instructional materials. Thus, in terms of quantified 
empirical data regarding variables such as frequency of use, satisfaction with 
materials, perceived difficulties of materials, and effects of materials upon 
student engagement and learning, we have a virgin territory for curriculum 
research. 

* 

Before any type of research can begin, a researcher must not only 
formulate a problem, he must make distinctions anmng the phenomena of interest 
that will enable him to pose questions worth asking. I believe it might be useful 
to distinguish between expressive materials which serve as heuristic devices for 
teaching or generating insights in a wide variety of areas within a curriculum and 
Instructional materials designed to achieve highly specific objectives. For example, 
a chalkboard or a map can be conceived of as expressive materials that have, in 
principle, an infinite array of curriculum uses in the classroom. Such materials 
can be used for generating modes of Inquiry 'whose products cannot be well 
predicted. Such devices are characterized by flexibility and by their ability 
to be used in ways that are not anticipated at the time they are designed or 
Invented. Cuisnaire Rods in contrast are more specialized. Although Cuisnalre 
Rods have great flexibility their use Is generally more circumscribed. They were 
invented for work within a particular field of study and are used to teach particular 
concepts, generalizations and operations. Color overlay boards or programmed 



frames tr e even more specific. These materials usually fit into a sequential 
program of study and teach highly specific skills, concepts, generalizations 
and so forth. Such materials are aimed at the achievement of specific 
terminal behavior. The point here is that curriculum materials can be 
designed to serve a range of uses from the very general and flexible-- a heuristic 
device would be an example— to the highly specialized instructional material* 
that is designed to zero in on facilitating highly t pacific learning. 

♦ 

The relevance of these distinctions become apparent when one seeks 
criteria for assessing their effectiveness. If one has designed instructional 
materials for achieving highly specific outcomes the evaluation problem is 
straightforward: one appraises the value of the materials by the extent to 
which they achieve what they were designed to achieve* If, however, one is 
attempting to evaluate expressive curriculum materials one needs to discover 
the consequences of their use, the range of activities in which they are employed 
and the manner in which both teachers and students work vjitb them. 1 cannot 
forget the experience I had observing a third grade girl viewing a single 
concept film developed to teach American children the firemaking skills of 
Australian aborigines. The small projector she monitored in her library 
carrel allowed her to stop the loop, rewind it, and repiay it. As she sat and 
watched a naked adolescent male aborigine build a fire she stopped the loop, 
called her girl friend over, and both had a lesson on the structure of male 
genlt&l*. What one hopes to teach, even when die materials are apparently 



pcclfic m focus, is not necessarily what Is learned. The evaluation of curriculum 
materials needs to take into account die ancillary, unanticipated outcomes of the 
use of such material. Hence, one coordinate that can bo identified to distinguish 
types of curriculum materials consists of those which are expressive and which 
serve as fle xib le heuristic devices related to the curriculum syUabii and those 
instructional devices which are considerably more limited or focused in the ways 
in which they can be used. 

Another coordinate distinguishes types of curriculum materials along 
the teacher-student use dimension. Some materials are designed exclusively 
for teachers to demonstrate —students view the demonstration by the teacher 
but do not experiment with the materials themselves. The group film is an 
example of such a device. Other instructional materials are designed for student 
manipulation and inquiry. Puzzle boxes, Guisnaire Rods, and the small simple 
microscopes that are provided in the ESS materials are examples of the latter. 

One could review the materials that have been developed to accompany written 
curriculum guides to determine the incidence of teacher demonstration to student 
manipulation materials and these types in relation to the expressive instructional 
continuum that has been outlined earlier. Whether a high incidence of any of 
these four types of materials make any difference in the educational process is 
net a question that at present can be answered, at least, by appealing to empirical 

a 

evidence. 

The evaluation of different types of curriculum materials will require the 
application of different types of criteria. Some of these criteria will require inspec- 
tion of the materials without necessarily examining their use in the classroom, Od|er 
criteria demand classroom observation and relate to how they are used, by whom, 



and their effects* 1 would now like to identify some of the criteria that seem to me 
to be appropriate for appraising curriculum materials. 

Perhaps the central question that can he asked of curriculum materials of 
an instructional type, that is, the type aimed at the attainment of specific objectives, 
is whether or not the content that 1$ offered is educationally significant* It is 
possible of course to develop devices that although attractive teach educationally 
trivial content. And while the significance of educations* content is to some degree 
contextual in character, in general, Judgments can be made about the import that 
the mr Serial is designed to teach. Determiniig the significance of such content is 

a first step in appraisal for unless the material is judged to be educationally sound 

♦ 

there is little reason to proceed with the application of other criteria. 

What constitutes adequate criteria for judging the significance of educational 
content will depend upon the expertise of specialists In the field from which the con** 
tent is drawn. The point is that judgments about the significance of the content that 
curriculum material is designed to teach is a primary element to be appraised in 
determining its adequacy. 

A second criterion that can be applied to appraise curriculum material 
Is one of judging its flexibility. By flexibility I mean the extent to which a given 
material can be used in different lessons or curriculum units. Materials which 

require the teacher to set up large and elaborate equipment to demonstrate one 

* 

brief idea will tend to be avoided by teachers. In our own curriculum development 
work At Stanford we have tried to avoid asking teachers to set up a projector, a 



screen and darken the classroom to show three or four slides to a class* The 
effort involved is not generally worth the reward. We have tried to design 
materials that could visually illustrate not only aspects of color or composition 

* 

in painting and drawing but which could be used to illustrate a variety of other 
visual elements as well. Such flexibility reduces the amount of material a 
school district must buy as well as the amount of space that classroom teachers 
must make available. Thus, ideally classrooms would have materials that were 
flexible in the sense that they could be used in a variety of ways and efficient 
in the sense that they could be used for sustained periods, 

A third criterion that can be applied to materials deals with the problem 
of producing materials that are durable. At Stanford we have had the experience 
of seeing second graders attempt to wash ink lines off of a plastic sheet attached 
tx> a cardboard backing. The ink lines, which were indeed washable; were to be 
wiped off with a damp sponge. Instead, the second graders took to more potent 
means and put them under the water tap. Needless to say what happened to the 
cardboard. We need to know how curriculum materials survive the rigors of 
classroom life and to find ways of extending that life whenever possible. When 

materials can be produced in consumable or non-consumable form 1 believe we should 

♦ 

opt for a non-consumabl e type in spite of the reduced profit to merchant or detignef* 
In the long run such economies might contribute to improved educational resources* 

A fourth criterion for appraising curriculum materials deals with the inci- 
dence of their use by both teachers and students in the classroom. Some of die 
research techniques that Mattel uses to test new toys might well be applied to 



M 9 M 



(vtluite new curriculum materials. Do the teachers and students use them? 

Do thfy enjoy u sin g them? Do they think they could be improved? If so* How? 

The fact th at a school or school district purchases curriculum materials is no 

guarantee that they will be used in the classroom. How much of such material 

gathers dust on the window shelf? When this happens how can the cause be 

determined? Is it due to the inherent unattractiveness of the material? Is it due 

to the fact that tt is inappropriate for the setting? is it due to a poor job of 

curriculum implementation? Whatever the reason* it is important to determine 

the incidence of use of curriculum materials. And to do that it will be necessary 

to either observe their use in the classroom or to establish some type of record 
• * 
geeping that will provide an accurate picture of how such materials are f i fact 

being used.* 

A fifth criterion that can be applied to appraise curriculum materials 

♦ 

deals with determining their ease of use. This criterion* logically* related to the 
previous one. deals with the demands that the materials make upon those who are 
• to use them. For example* the use of alligator clips on materials young elementary 
school children are to use must be related to their ability te handle such clips. In 
our own project at Stanford we found that the stacking of art reproductions for 
storage caused some elementary school teachers to cut their fingers when putting 
them out. It is unlikely that teachers will continue to use ma terials under such 
circumstances. We need to know how accessible these materials are and need 
to develop ways of finding out. If it is true that most of the materials that are 
designed to accompany curriculum guides are used only marginally it becomes 
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Intriguing to find out why and to remedy the situation, 



after all of these types of criteria arc applied to appraise 
the quality, use, and satisfaction of curriculum materials die central question 
that must be asked of all educational inn ovations remains: Doe3 it help 



educate those who use them? At present answers to this question from the 
standpoint of research are difficult to obtain. I would suggest that at least one 
way to answer such a question is to attempt to appraise the effects of curriculum 
materials wholistically. That is, to view their effects within the context of the 
total program. This approach is similar to determining the effects of a visual 
element in a pattern in the context of the pattern rather than by removing the 
element from the pattern and examining it independently. While the latter type 
of examination is appropriate for the application of some types of criteria (types 
which I have already described) the problem of judging the effects of materials 
needs, I believe, to be examined more organically since the effects curriculum 
materials have are determined in part by their interaction With other elements of 
the curriculum. Their appraisal therefore should, at least in part, be made in 
the context in which they function. To no this well will call for research methods 
that differ markedly from the types used in classical experimentation. To the 

fields of art criticism and cultural anthropology we might look to find new and 

• • 

more appropriate ways of. appraising the tools we use to educates. 
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